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Self-normalized processes arise naturally in statistical applica- 
tions. Being unit free, they are not affected by scale changes. More- 
over, self-normalization often eliminates or weakens moment assump- 
tions. In this paper we present several exponential and moment in- 
equalities, particularly those related to laws of the iterated logarithm, 
for self-normalized random variables including martingales. Tail prob- 
ability bounds are also derived. For random variables Bt> and At, 
let Yt(X) = expjAylt — A 2 B 2 /2}. We develop inequalities for the mo- 
ments of A t I 'B t or sup t>0 At /{B t ( log log At) 1 / 2 } and variants thereof, 
when EYt(X) < 1 or when Yj(A) is a supermartingale, for all A be- 
longing to some interval. Our results are valid for a wide class of ran- 
dom pr ocesse s including continuous martingales with At = Mt and 
B t = y/{M)t, and sums of conditionally symmetric variables di with 

At = ^2* i=1 di and Bt = a^/V*^ rff ■ A sharp maximal inequality for 
conditionally symmetric random variables and for continuous local 
martingales with values in R m , m > 1, is also established. Another 
development in this paper is a bounded law of the iterated logarithm 
for general adapted sequences that are centered at certain truncated 
conditional expectations and self-normalized by the square root of 
the sum of squares. The key ingredient in this development is a new 
exponential supermartingale involving J2-=i ^ an< ^ S'=i ^ com- 
pact law of the iterated logarithm for self-normalized martingales is 
also derived in this connection. 

1. Introduction. A prototypical example of self-normalized random vari- 
ables is Student's i-statistic which replaces the population standard devi- 
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ation a in the standardized sample mean y/n(X n — p)/cr by the sample 
standard deviation. More generally, a self-normalized process is of the form 
At/Bt, in which Bt is a random variable that estimates some dispersion mea- 
sure of At- An important aspect of the theory of self- normalized processes 
is that we can often dispense with the moment conditions that are needed 
if At is normalized by nonrandom bt instead, as evidenced by Shao's (1997) 
large deviation theory for self-normalized sums of i.i.d. random variables 
without moment conditions. The problem of moment inequalities for self- 
normalized processes was suggested to the first author in 1990 by J. L. Doob, 
who pointed out that a key open problem in martingale theory was the de- 
velopment of inequalities for martingales that are analogous to known results 
in harmonic analysis [see Bahuelos and Moore (1999) for results in this di- 
rection] . 

In recent years, there has been increasing interest in limit theorems and 
moment bounds for self-normalized sums of i.i.d. zero-mean random vari- 
ables Xj. In particular, Bentkus and Gotze (1996) derive a Berry-Esseen 
bound for Student's t-statistic, and Gine, Gotze and Mason (1997) prove 
that the i-statistic has a limiting standard normal distribution if and only 
if X\ is in the domain of attraction of a normal law, by making use of ex- 
ponential and Lp-bounds for the self-normalized sums U n = S n /V n , where 
S n = Ya=i Xi and V£ = Ya=i Xf- Egorov (1998) gives exponential inequali- 
ties for a centered variant of U n . To see the connection between the i-statistic 
T n and the self-normalized sum U n , observe that 



(1.1) 



Sn/V n 



n - (S n /V n ) 2 }/(n - 1) 

A recent paper of Caballero, Fernandez and Nualart (1998) contains moment 
inequalities for a continuous martingale over its quadratic variation and uses 
these results to show that if {Mt,t > 0} is a continuous martingale null at 
zero, then for each 1 < p < q, there exists a universal constant C = C(p, q) 
such that 



(1.2) 



Mt 



(M)t 



<C 



1 



(M) 



1/2 



Related work in Revuz and Yor [(1999), page 168] for continuous local mar- 
tingales establishes for all p > q > the existence of a constant C pq such 
that 



(1.3) 



E- 



(sup s<00 |M s |)P 



(M)t 2 



< C pq E[ sup |M, 



p-q 



It is important to point out that neither (1.2) nor (1.3) provide bounds 
for what is arguably the most important case of inequalities of this type, 
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namely p = q. Bounds on E(\Mt \ p / (M)t ) are of particular interest because 
of their connection with the central limit theorem, as noted earlier in the case 
of self-normalized sums of i.i.d. random variables. For discrete-time martin- 
gales {^27=1 di, Tn, n > 1}) de la Peha (1999) provides exponential bounds 
for the tail probabilities of £™ =1 ck / '(a + (3V 2 ) , where V 2 = Y2=i E (<% l-^-O 
and /3 > 0, a > 0. In view of the law of the iterated logarithm (LIL), it 
is of interest to use V n or V n y/2\og log V n (instead of V 2 ) to self-normalize 

£2=1*. 

Motivated by these developments, we establish in this paper analogous 
exponential and L p -bounds for a martingale divided by the square root of 
its quadratic variation or its conditional variance. We start by considering 
two random variables A and B with B > such that 

f A 2 1 

(1.4) EexplXA B 2 \ <1 for all A G R. 

Note that if we were allowed to maximize over A inside the expectation, then 
the maximizing value A = A/B 2 would give us I?exp(^4 2 /21? 2 ) < 1, which 
in turn would imply that P(A/B > x) < exp(— x 2 /2). Although we cannot 
interchange the order of max^ and E, we can integrate over A with respect 
to a probability measure F and interchange the order of integration with 
respect to P and F. This approach is used in Section 2 to derive not only 
tail probability bounds for A/B but also L p and exponential bounds for 
A/^/B 2 + (EB) 2 , and in Section 3 to obtain iterated logarithm bounds for 
the moments of A + /B. Section 3 further extends the results to the case 
where (1.4) is replaced by 

(1.5) Eexp{\A-$(\B)}<c for all < A < A , 

in which $ is assumed to be any nonnegative, strictly convex function on 
[0,oo) such that ^(O) = 0, lim x ^ 00 $(x) = oo and limsup^go <3?"(x) < oo. 
Important special cases of such <I> are & r (x) = x r /r with 1 < r < 2. 

We next replace the random variables A and B by random processes 
A t and B t and, accordingly, replace (1.5) by 

(1.6) {exp(Aj4( — & r (\B t )), t G T} is a supermartingale for all < A < Ao, 

in which T is either {0, 1,2,...} or [0, oo). Section 4 proves an expectation 
form of the LIL (Theorem 4.1) and develops maximal inequalities under 
this assumption. Moreover, the case r = 2 and Ao = oo in (1.6) with "super- 
martingale" replaced by "martingale" yields a formula for certain boundary 
crossing probabilities of continuous local martingales taking values in R m , 
as shown in Corollary 4.3. Motivated by the LILs for self-normalized sums 
of certain classes of i.i.d. random variables due to Griffin and Kuelbs (1989, 
1991), Shao (1997) and Gine and Mason (1998) and extensions by Jing, 
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Shao and Wang (2003) to sums of independent zero-mean random variables 
satisfying a Lindeberg-type condition, we study almost sure LILs for self- 
normalized (discrete-time) processes in Sections 5 and 6. 

When a partial sum of random variables X\ , X<i ,.. . is centered and nor- 
malized by a sequence of constants, only under rather special conditions does 
the usual LIL hold even if the variables are i.i.d. In contrast, we show in Sec- 
tion 5 that there is a universal upper bound of LIL type for the almost sure 
rate at which such sums can grow after centering by a sum of conditional 
expectations of suitably truncated variables and normalizing by the square 
root of the sum of squares of the Xj's. Specifically, let S n = X\ + ■ • ■ + X n 
and V 2 = X\ + • • • + X 2 , where {Xi} is adapted to an increasing sequence 
{J^i} of cr-fields. In Section 5 we prove that given any A > 0, there exist 
positive constants a\ and 6a such that liniA^+o = and 

(1.7) limsup< S n - ^2^i(-\v n ,a x v n ) W{V n (loglogT/ n ) 1/2 } < b\ a.s. 

n^oo L . =1 J/ 

on {limV^ = oo}, where v n = V n (loglog V n )~ 1 / 2 and /U,(c,d) = E{Xit(c < 
Xi < d^Fi-i} for c < d. Note that (1.7) is "universal" in the sense that it is 
applicable to any adapted sequence {Xi}. In particular, Suppose {S n , J~ni fl ^ 
1} is a super martingale such that X n > — m n a.s. for some J^-i-measurable 
random variable m n satisfying P{0 < m n < \v n for all large n} = 1. Then 

(1.7) yields 

(1.8) limsup5 n /{14(loglogy n ) 1 ^ 2 } < 6a a.s. on {lim V n = oo}. 

We derive in Section 6 the lower half counterpart of (1.8) for the case 
where {S n ,!F n ,n > 1} is a martingale such that \X n \ < m n a.s. for some 
JT n „ 1 -measurable m n with v n — ► oo and m n /v n — > a.s. Combining this 
with (1.8) (with liniA^o&A = V%) then yields 

(1.9) limsu P 5 n /{K(loglogK) 1/2 } = V / 2 a.s. 

We end this section with various lemmas identifying a large class of ran- 
dom variables satisfying (1.4), (1.5) or (1.6). 

Lemma 1.1. Let Wt be a standard Brownian motion. Assume that T is 
a stopping time such that T < oo a.s. Then E expjAWr — A 2 T/2} < 1 for all 
AeR. 

Lemma 1.2. Let Mt be a continuous, square-integrable martingale, with 
M = 0. Then 

(1.10) exp{AM t -A 2 (M) i /2}, t > 0, is a supermartingale for all A G R. 

If Mt is only assumed to be a continuous local martingale, then (1.10) is 
also valid (by application of Fatou J s lemma). 
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Lemma 1.3. Let {Mt:t>0} be a locally square-integrable martingale, 
with Mq = 0. Let {Vt} be an increasing process, which is adapted, purely 
discontinuous and locally integrable; let be its dual predictable projec- 
tion. SetX t = M t + V t , C t = £ s < t ((AX s )+) 2 , D t = {£ s < 4 ((AX s )-) 2 }[ p) , 

H t = (M)t + Ct + D t . Then exp{X t — vj^ — \H t } is a supermartingale and, 
hence, 

(1.11) Eexp{\{X t - V t {p) ) - X 2 H t /2} < 1 for all A e R. 

Lemma 1.3 is taken from Proposition 4.2.1 of Barlow, Jacka and Yor 
(1986). A related bound can be found in Lemma 1.5, due to Stout (1973), 
in which At is a discrete-time martingale with bounded increments and 
B 2 is a multiple of its conditional variance; see also Kubilius and Memin 
(1994). The following lemma holds without any integr ability conditions on 
the variables involved. It is a generalization of the fact that if X is any 
symmetric random variable, then A = X and B = X 2 satisfy condition (1.4). 
It has a long history, including Wang (1989) and Hitczenko (1990). Hitczenko 
(1990) proved it for conditionally symmetric martingale difference sequences, 
and de la Peha (1999) pointed out that the same result still holds without 
the martingale difference assumption and, hence, without any integrability 
assumptions. 

Lemma 1.4. Let {di} be a sequence of variables adapted to an increasing 
sequence of a -fields {J~i\- Assume that the d^s are conditionally symmetric 
[i.e., C{d i \T i -x)=C{-d i \T i - 1 )). T/i e nexp{AEti^-A 2 Er=i^/2}, n> 
1, is a supermartingale with mean < 1, for all A € R. 

Note that any sequence of real-valued random variables Xi can be "sym- 
metrized" to produce an exponential supermartingale satisfying (1.8) by 
introducing random variables X- such that 

• • • i X n -\, X' n _ 1 , X n ) = C(X n \Xi, . . . ,X n -i) 

and setting d n = X n — X' n ; see Section 6.1 of de la Peha and Gine (1999). 
The next two lemmas are related to (1.6). 

Lemma 1.5. Let {d n } be a sequence of random variables adapted to an 
increasing sequence of a-fields {F n } such that E(d n \J r n -i) < and d n < 
M a.s. for all n and some nonrandom positive constant M . Let < Ao < 
M-\ A n = ELl dt, B 2 n = (l + iA M) Ya=i E(d 2 \Fi-i), A = B = 0. Then 
{exp(AA n — ^X 2 B 2 ), F n , n > 0} is a supermartingale for every < A < Aq. 
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Lemma 1.6. Let {d n } be a sequence of random variables adapted to an 
increasing sequence of a-fields {J- n } such that E(d n \J- n -i) = and o~\ = 
E(d\ \J~n-i ) < °°- Assume that there exists a positive constant M such that 
E{\d n \ k \F n -i) < (k\/2)orlM k - 2 a.s. or P(\d n \ < M\F n -i) = 1 a.s. for all 
n > 1, k > 2. Let A n = £? =1 V% = £™ =1 E(d 2 \T i _ 1 ), A = V = 0. Then 

2{1-MX) ■ 

X< 1/M. 



{exp(A^4 n — 2(i-M\) ^ 2 Vn )) F n , n > 0} is a supermartingale for every < 



Fix any < p < 1. Then Lemma 1.6 implies that (1.4) holds with A = 
A n and B = V n /yfp for every < A < (1 — p)/M. The supermartingale in 
Lemma 1.6 is closely related to martingale extensions of the classical in- 
equalities of Bernstein and Bennett; see Section 8.3 of de la Peha and Gine 
(1999) for a unified approach to developing such inequalities from corre- 
sponding results for sums of independent random variables via decoupling. 

2. Some exponential inequalities. In this section we present a simple 
method to derive exponential and L p -bounds for Aj \J B 2 + (EB) 2 under 
assumption (1.4). 

Theorem 2.1. Let B > and A be two random variables satisfying (1.4) 
for all A G R. Then for all y > 0, 

(2.1) E , V exp(— -4 1 orl<l. 

1 ' JWTy 1 P \2(B 2 + y 2 )j- 

Consequently, if EB > 0, then Eex.p(A 2 /[A(B 2 + (EB) 2 ]) < ^2 and 



(2.2) Eexp(xA/\/B 2 + (EB) 2 ) < V2 exp(x 2 ) for all x > 0. 
Moreover, for all p> 0, 



(2.3) E(\A\/VB 2 + (EB) 2 f < 2 p ~ 1 l 2 pT(p/2). 

Proof. Multiplying both sides of (1.4) by (27r)~ 1 / 2 y exp(-A 2 y 2 /2) (with 
y > 0) and integrating over A, we obtain by using Fubini's theorem that 

1> r E^e,JxA-^B 2 )e,J-^-)dX 



E 



oo v 2ir \ 2 J \ 2 

A 2 

■. exp 



"U(B 2 +y 2 ) 

exp(-^ (X 2 - 2^X + } dX] 



V2^ *H 2 V B 2 +y 2 ' (B 2 + y 
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proving (2.1). By Schwarz's inequality and (2.1), 
A 2 

Eexp 



i(B 2 +y 2 ) 



E y^{A 2 /(2(B 2 + y 2 ))} \ f jB 2 + y 2 



VB 2 + y 2 )\ V V 



1/2 



,2 



+ <(e(- + i)) / <V2 for y = EB. 



\ v y 2 J \ \y 

To prove (2.2) and (2.3), we assume without loss of generality that EB < oo. 
Using the inequality \ab\ < ^±^, with a = y/TcA/^jB 2 + {EB) 2 and b = 
x/y/2c, we get xA/y'B 2 + (EB) 2 < ^T^mm 2 + wnich in the case c= 1/4 
yields 

Mti^W} s Mi^w + £ } - ^ exp(a:2, • 

proving (2.2). Moreover, by Markov's inequality, P(\A\/yjB 2 + (EB) 2 > 
x) < \/2exp(— x 2 /4) for all x > 0. Combining this with the formula EU P = 
Jo° px p ~ 1 P(U > x) dx for any U > 0, we obtain 

E(\A\/VB 2 + (EB) 2 ) P <V2 / px p - 1 exp(-x 2 /4)dx = 2^ 1 /V(p/2). 

Jo 

□ 

Another application of the basic inequality (2.1) is the following. 

Corollary 2.2. Let B > and A be two random variables satisfying 
(1.4) for all A G R. Then for all x > \/2, y>0 and p>0, 



(2.4) p\\A\/^(B 2 + y)(l + ^log(^ + lj^>xj<exp K g 

(2.5) E (\A\ /^(B 2 + y) (l + ± log + l) ) J < V>'* + 2<P~WpT 



PROOF. Note that for x > y/2 and y > 
A 2 x 2 / 1 /5 2 

.^^)-t1 1+ 2 1o <t 4 
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A 2 x 2 1 (B 2 1 

x 2 \^e W {A 2 /(2(B 2 +y))} ( x 2 



^{-Y) E VWTy - eXP l"T 

in which the last inequality follows from (2.1). The proof of (2.5) makes use 
of (2.4) and is similar to that of (2.3). □ 

3. Iterated logarithm bounds for moments of self-normalized variables 
and their generalizations. In this section we present bounds for Eh(A + / B) 
in terms of E{H(B)} , where H is a function that depends on h. The basic 
results are Theorems 3.3 and 3.6. Applications of these results are given 
in Examples 3.4, 3.5 and 3.8, which relate, in particular, the pth. absolute 
moment of A + /B to that of the iterated logarithm Vloglog^ V B~ l V e 2 ). 
A variant of Theorem 3.3 has been derived by a different argument in The- 
orem 1 of de la Peha, Klass and Lai (2000) and Lemmas 3.1 and 3.2 below 
provide the proofs of Lemmas 2 and 3 of that paper. The main objective 
of this section is to develop an analogous result that requires (1.4) to hold 
only for the restricted range < A < Ao, thereby widely expanding the ap- 
plicability of our approach. In particular, this extension (given in Theo- 
rem 3.6), together with Lemma 1.5, provides moment bounds for a wide 
class of discrete-time martingales self-normalized by the square root of the 
conditional variance, thereby connecting our results to LILs. Stout (1973) 
and Einmahl and Mason (1989) have used this type of self-normalization for 
LILs of martingales. 

Let L : (0, oo) — ► (0, oo) be a nondecreasing function such that 

(3.1) L(cy) < 3cL(y) for all c > 1 and y > 0, 

(3.2) L(y 2 ) < 3L(y) for all y>l, 

(3 ' 3) / ^L{x) = 2- 

An example satisfying (3.1)-(3.3) is the function 

(3.4) L(y) = 0{\og(y + a)}{log log(y + a)}{log log log(y + a)} 1+s , 

where 5 > 0, a is chosen sufficiently large to ensure (3.1), (3.2) and j3 is a 
normalizing constant so that (3.3) holds. 

Lemma 3.1. Let^>l. Then yL(y/B\/ B/y) <3~{{L(~/)V L(B\/ B~ 1 )} 
for any < y < 7 and B > 0. Consequently, for any A> B > and any 
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Proof. First consider the case y < 1. From (3.1) and the fact that L is 
nondecreasing, it follows that 

For the remaining case 1 < y < 7, since L is nondecreasing, we have 
(3.6) 

< 7 {^(7 2 ) VL((Bvi) )}< 3 7 {l( 7 ) VL^vij), 

where the last inequality follows from (3.2). □ 

Lemma 3.2. Lei -B > and A be random variables satisfying (1.4) /or 
all X> 0. Define 

(3.7) ^) = ^^l 1( x>l). 



Then 



E ,,, 9 ^ , < 



L(A/B) V L(5 V l/B) ~ ft exp(-x 2 /2) dx ' 

PROOF. By a change of variables, J 1 (AL(l/A))~ 1 dA = J™ (XL(X))- 1 d\ = ±. 
Let 

( 3 - 8 ) /( A ) = TT7 77—77777, A>0. 

v ; Jy ' AL(max{A, 1/A}) 

Then J °° / (A) d\ = Jq 1 /(A) dX + J^ 00 / (A) d\ = 1, so / is a density function on 
(0,oo). Therefore, integrating (1.4) with respect to this probability measure 
yields 

Jo xL{xVl/x) 

„ f°° exp{Ay/B - (y 2 /2)} , „ . „ , 

= E L yLiy/BVB/y) ^ Q*to*V = B*) 

{ A 2 \ 



> E{ exp 



\2& 



ex P {-(, 2 /2)} ,M >1)dx 



A/B (x + A/B)L({(x + A/B)/B} V {5(x + A/5)}) V# 



A 

letting x = y — — 
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; { exp Q&)}/° 



[by (3.5)] 

1 x 2 \.Av 9(A/B) 



L{A/B) VL(BV1/B)' D 



We next derive a bound on Eh(A + / B) by making use of Lemma 3.2 for 
nondecreasing functions h that do not grow faster than g/L. 

Theorem 3.3. Let L : (0, oo) — > (0, oo) be a nondecreasing function sat- 
isfying (3.1)-(3.3). Define g by (3.7). Let h be a nondecreasing function on 
[0, oo) such that for some xq > 1 and c > 0, 

(3.9) < h(x) < cg(x)/L(x) forallx>XQ. 

Let q be a strictly increasing, continuous function on [0, oo) such that for 
some c>c, 

(3.10) L{x)<q{x)< C j^ forallx>x . 

Let B > and A be random variables satisfying (1.4) for all A > 0. Then 

(3.11) Eh(A + /B) < Ac + h(x ) + Eh{q~ x {L{B V B^ 1 ))). 
Consequently, Eh(A + /B) < oo if Eh(q' l {L(B V B' 1 ))) < oo. 

PROOF. By Lemma 3.2, 

g(A+/B) 
E — — < 4. 

L(A/B)VL{BV1/B) ~ 

Let Q = {L(B Vi)< q(%)}. Then, Eh(A + /B) is majorized by 
h(A+/B)l(Q)l(A/B > xq) 



h{x ) + E 



g{A/B)/(L{A/B)VL(BVl/B)) 

9{A/B) w£w(4>*«. 



L(A/B)V L(BV1/B)J \B J \B 

h(y)(L(y)Vq{y)) 



< h(x ) + sup 

y >x 9{y) 

< ft(*o) + 4 sup + Eh ( q -i ( L ( B v I 
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□ 



To apply Theorem 3.3, one can take L as given by (3.4) and choose q" 1 
that grows as slowly as possible (or equivalently, q that grows as rapidly as 
possible) subject to the constraint (3.10). 

Example 3.4. Define L by (3.4) and let h(x) = x p for x > 0, with p > 0. 
Then (3.9) clearly holds with c = 1 and xq sufficiently large, for which (3.10) 
also holds with q{x) = g(x)/h(x) = exp(x 2 /2)/x p+1 . In this case, 

(T l {y) ={21ogy+ (p + l + o(l))loglogy} 1/2 as y^oo. 

Since L(x) ~ /3(logx)(loglogx)(logloglogx) 1+<5 as x — > oo, Theorem 3.3 yields 

(3.12) E(A + /Bf < oo if ^{logdlog^V^ 1 )! Ve)} p/2 <oo, 

for random variables B > and A satisfying (1.4) for all A > 0. 

Example 3.5. Let < 9 < 1 and h(x) = exp(9x 2 /2) for x > 0. Define 
L by (3.4). Then (3.9) holds with c = 1 and xq sufficiently large, for which 
(3.10) also holds with q{x) = g(x)/h(x) = x~ l exp{(l — 6)x 2 /2}. In this case, 
hiq^iy)) = Odyilogy) 1 / 2 } 6 ^ 1 ^). Therefore, if B > and (1.4) holds for 
all A > 0, then by Theorem 3.3, 



for some 5 > 0, where B = B\J B^ 1 V e 3 . 

The following theorem modifies and broadly extends Theorem 3.3 by re- 
quiring (1.4) to hold only for the restricted range < A < Ao- An example 
where this appears naturally can be found in Lemma 1.5, where A is a 
martingale and B 2 is a multiple of its conditional variance. Theorem 3.6 
also generalizes (1.4) by replacing the quadratic function X 2 B 2 /2 and the 
upper bound 1 in (1.4) by a convex function &(AB) and a finite positive 
constant c. Unlike Theorem 3.3 that involves a single function q to give the 
bound (3.11), Theorem 3.6 uses a family of functions qb- The wider range 
of applications that will be explored in Section 4 justifies the additional 
technical work required for the theorem. The proof employs different anal- 
yses of A/B for small and large B, incorporating a Taylor expansion of $ 
for large B. In addition, as before, Fubini's theorem allows us to treat the 
random variables involved as though they were constants. 



(3.13) 




if ^(logBJOoglogSj^CIogloglogS) 1 - 1 -'}^ 1 -^ < oo 
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Theorem 3.6. Suppose that $(•) is a continuous function with $>'(x) 
strictly increasing, continuous and positive for x > 0, with lim x ^ 00 ^(x) = oo 
and sup a;> Q <&"(x) < oo. Suppose B > and A are random variables such that 
there exists c> for which 

(3.14) £exp{AA- <5>{\B)} < c for all < A < A . 
For w > 3>'(1), define y w by the equation <&'(y w ) = w, and let 

(3.15) g$(w) =y~ 1 ex.p{wy w - $(y w )}. 

Let rj > fj > 0. Let h : [0, oo) — ► (0, oo) be a nondecreasing function. For b>rj, 
let qb be a strictly increasing, continuous function on (0, oo) such that for 
some c > and wq > $'(2), 

96 M < c{g$(w)t(y w < X b) + e x ^ w t(y w > \ b)}/h(w) 

(3.16) 

for all w >wq- 

Let L:(0,oo) — > (0,oo) be a nondecreasing function satisfying (3.1)-(3.3). 
Then there exists a constant C depending only on \o,r),fj,c,c and $ such 
that 

(3.17) Eh(A + /(B V V ))<C + h(w ) + Eh(q£jL{B V rj))). 



Proof. Note that Lemma 3.2 transforms the inequality constraints 
(1.4) for all A > into a single expectation inequality primarily involving 
a rather rapidly growing function of A/B and a slowly varying function L 
of BV This result is employed in the proof of Theorem 3.3 to bound a 
quantity of the form Eh(A + /B) by a constant plus Eh(q~ 1 (L(B V -g)). To 
duplicate this approach when (1.4) holds only for < A < Ao, we first derive 
an analog of Lemma 3.2 by splitting A/B > wq into two cases: yA/B > Aq-B 
and yA/B Ao-B. Moreover, we need to replace B by B V r]. Since $>(x) is 
increasing in x > 0, (3.14) also holds with B replaced by B V i] and, there- 
fore, we shall assume without loss of generality that B>rj. Integrating (3.14) 
with respect to the probability measure defined by the density function (3.8) 
yields 

V ; ~ Jo AL(AVA-!) Jo xL(x/BVB/x) 

Our first variant of Lemma 3.2, given in (3.19), provides an exponential 
bound for A/B when Ao-B < Va/b- Observe that using the definition of y w , 
we have that x^ — $(x) increases in x for x < yA/B: an d decreases in x for 
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x > ua/b- Take any < fj < rj, and let Ai = Ao V A 1 V r). Since B > rj > fj, it 
follows from (3.18) and (3.1) that 



c>E 



(3.19) >E 



x °v exp{xA/B - $(x)} (A 



xL{x/B V B/x) 



dxt 



B 



> w Q \l(y A / B > A -B) 



x ov exp{A r?A/£? - $(A r/)} dx f A 

r /x x/ 1 ^ > HVa/b > MB) 

x rj I(A V5/(A i))) x \B 



e -*(A ^) 

> : ; log 



3Ai/»7 



E 1 ( - > w ) 1(j/a/b > Ao-B). 



L{B) 



Our second variant of Lemma 3.2, given in (3.21), bounds A/B when 
Aq-B > Ua/b- Since wq > <&'(2),y W0 > 2. Define 



(3.20) 



a* = sup{a < 1 : a ^"(x) < 1 for all x > y WQ — a}. 



Note that a* > and y Wo — a* > 1. Since $'(?/«,) — w = 0, a two-term Taylor 
expansion for w > and x G (y,„ — a*,y w ) yields 



wx - $(x) = wy w - $(y u 
> wy w - $(y 



(x y w ) 2 



(x - y w f 
2al 



in which £* lies between x and y^,. The last inequality follows from (3.16) 
and (3.20), noting that £* > x > y w — a* > y WQ — a*. It then follows from 
(3.18) that 



c> E 



A/B < Ao-B, "g > ^0 

y^/B exp{(A/B)y A / B ~ Hva/b) -{x- y A /B) 2 /{1al)} 



VA/B~ a * 



xL{xjB\l B/x) 



dx 



> E 



A 



^[va/b < Aq-B,— >w 



exp{(A/B)y A/B - <Z>{y A / B )} [ yA ' B 



Va/b{L(\v V B)} 



exp 



VA/B-a* 



using x > yA/B ~ a * > 2/wo — a* > 1 so that ^ < B. From Lemma 3.1 and the 



Va/b) 



2al 



dx 



fact that B>rj,we have L(A VB)< 3(1 V ^)L(S). Hence, 



c> E 
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(3 91) : , <*P{WB)VA,B-*(VA,B)} 
(3 ' 21) 3y A/B (lV(X /v))L(B) °* Jo 2 ) d " 

a* g$(y A/B )t(y A/B <\ B,A/B>w ) 

-4(lV(A /77)) L(B) 

Let Q = {L(B) < q B (A/ B)} . Then rewriting (3.16) as an upper bound for 
h and using the definition of Q, we can majorize Eh(A + j B) by 

h(wo) + cE 



HQ){^^t^> W0 ,y A/B <\ B 

e \ f]A/B / 

+ jjg; > wo,y A/B > \ B 



<h(wo) + C + Eh(q^(L(B))), 
in which the inequality follows from (3.16), (3.21) and (3.19). □ 

Remark 3.7. In the case Xq = oo [as in Theorem 3.3 for which $(x) = 
x 2 /2], the bounds (3.18) and (3.19) are not needed and the result for gen- 
eral $ is similar to (3.11) in Theorem 3.3. The main difference between 
(3.11) and (3.17) lies in q~ l in (3.11) versus the more elaborate q B ^„ in (3.17) 
to incorporate both (3.19) and (3.21). 

The next example is designed to exploit the form of qb(w) of Theorem 3.6 
[see (3.16)]. 

Example 3.8. Lemmas 1.5 and 1.6 give examples of (A,B) satisfying 
(1.4) only for < A < A . Thus, (3.14) holds with $(x) = x 2 /2 and g<s> reduces 
to the function g defined by (3.7) in this case, noting that y w = w. Define L 
by (3.4). First let h{x) = x' p for x > 0, with p > 0. For b > rj > fj > 0, let qb 
be a strictly increasing function on (0, oo) such that for all large b, 

q b ( w ) = e w2 / 2 / W P +1 ifw< A (??6) 1/2 , 

(3.22) <e w2 / 2 /wP+ l if A (??6) 1/2 <u;< A 6, 

= e Xo ^ w /w p ifw> X b. 

Then (3.16) holds with 5=1. From (3.4) and (3.22), it follows that q^ l (L{b)) - 
(2 log log 6) 1 / 2 as b — > oo. Therefore, (3.12) still holds with B replaced by 
B V 7] even though (1.4) holds only for < A < Ao- Similarly, letting h(x) = 
e^ x with < C < Ao??, it follows from Theorem 3.6 that 

Eexp((A + /(B Vn)) < oo 

(3 23) 

if E exp{C [2(log log B) (log log log B) 1+s ] 1/2 } 
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for some 5 > 0, where B = BV e 3 . One such choice of (ft that satisfies (3.22) 
for sufficiently large b is to let qb(w) = w~ p exp(/ 2 (w)) for Ao(^fr) 1 / 2 < w < 
Ao6, where / is linear on [\o{rjb) 1 / 2 , Ao6] and is uniquely determined by 
requiring % to be continuous. In this case, it can be shown that f 2 (w) < 
w 2 /2 — logw for Ao(f?6) 1 ^ 2 < w < Ao& if & is sufficiently large, noting that the 
slope of / is {1 + o(l) — l/y/2}^fj/b and, therefore, \w 2 — logw — f 2 (w) is 
an increasing function of w G [Xo^rjb) 1 / 2 , Xob] for all large b. 

Another application of Theorem 3.6 involves the more general case of 
<3?(x) = x r jr (1 < r < 2), for which 

(3.24) y w = U ; 1 /(r-l) 5 = ^-l/Cr-l) exp {(1 _ r -l) w »-/(r-l)}_ 

In view of (3.24), it follows from Theorem 3.6, by arguments similar to 
Example 3.8, that under (3.14) with $(x) = x r /r, we have for any p > 0, 

(3.25) E(A + /(BV V )) P <oo if £{log+(log(5 V r ? ))f( r ~ 1 )/ r < oo. 

Moreover, (3.23) still holds if we replace 2 and 1/2 there by r/(r — 1) and its 
reciprocal, respectively. The following lemma, which provides an analogue 
of Lemma 1.5 for more general 1 < r < 2 and which self-normalizes A n by 
the square root of the square function J2?=idj, gives an exponential super- 
martingale when the summands di of A n are bounded from below rather 
than from above. 

Lemma 3.9. Let 0<7<l<r<2. Define c 7jr = max{c r , cf 7 "*}, where 
c r = inf{c > : exp(x — cx r ) < 1 + x for all x > 0}, 
4 7 ^ = inf{c > : exp(x — c|x| r ) < 1 + x for all — 7 < x < 0}. 

(i) For all x> —7, exp{x — c 7jr |x| r } < 1+x. Moreover, c r < (r — l) r_1 (2 — 
r) 2 ~ r jr and 

00 

47) = _ {7 + io g (i _ 7 )}/ 7 f = £ 

i=2 

(ii) Let {d n } be a sequence of random variables adapted to an increasing 
sequence of a -fields {J- n } such that E{d n \!F n —x} < and d n > —M a.s. for 
all n and some nonrandom positive constant M. Let A n = Y^i=\di, B r n = 
rc 7,rZ)r=i \di\ r , ^0 = Bq = 0- Then {exp(A^4 n — (XB n ) r /r), T n ,n > 0} is a 
supermartingale for every < A < 7-M -1 . 

Proof. The first assertion of (i) follows from the definition of c 7>r . 
For c > 0, define g c (x) = log(l + x) — x + c\x\ r for x > —1. Then g' c (x) = 
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\x\ r 1 {\x\ 2 r (l — |x|) 1 — cr} for —1 < x < 0. Since \x\ 2 r /(l — |x|) is de- 
creasing in — 1 < x < 0, g' c has at most one zero belonging to (—1,0). Let 
c* = -{7 + log(l - 7)}/7 r . Then g c *(-i) = = g c *(0). It then follows that 
g c *(x) > for all —7 < x < and, therefore, c* > of \ If c* > Cr 7 \ then 
g ( 7 ) (—7) < g c * ( — 7) — 0, contradicting the definition of cf 7 "* . Hence, cl 7 ^ = c* . 
Take any c > (r - l) r " 1 (2 - r) 2 ~ r jr. Then for all x > 0, 



Since <? c (0) = 0, it then follows that g c (x) > for all x > 0. Hence, c r < 
(r-l) r ~ 1 (2-r) 2 ~ r /r. 

To prove (ii), note that since Xd n > —AM > —7 a.s. for < A < 7M -1 , 
(i) yields 

E[exp{A«i n - c 7ir |Ad n | r }|JP" n _i] < E[l + A^l^-i] < 1 a.s. □ 

4. An expectation version of the LIL and maximal inequalities for self- 
normalized martingales. In this section we first prove a theorem that pro- 
vides an expectation form of the upper LIL under the assumption 



where T is either {0,1,2,...} (discrete-time case) or [0,oo) (continuous- 
time case) and $ r (x) = x r jr for 1 < r < 2. Applications of the theorem 
will be given in (4.9)-(4.12). Important special cases of (4.1) have been 
given in Lemmas 1.5, 1.6 and 3.9. We then develop maximal inequalities 
for self-normalized processes under (4.1), yielding an almost sure upper LIL 
in Corollary 4.2 that generalizes a corresponding result of Gine and Mason 
(1998) for i.i.d. symmetric random variables. 

Theorem 4.1. Let T = {0,1,2,...} orT = [0,oo), 1 < r < 2, and<5> T (x) = 
x r /r for x > 0. Let A t ,B t be stochastic processes (on the same probability 
space) satisfying (4.1) and such that Bt is positive and nondecreasing in 
t > 0, with Aq = 0. In the case T = [0,oo), assume furthermore that At and 
Bt are right- continuous. Let L : [1, 00) — > (0, 00) be a nondecreasing function 
satisfying (3.1)-(3.3). Let n > 0, Ao^ > e > 0, and h: [0, 00) — > [0, 00) be a 
nondecreasing function such that h(x) < e £X for all large x. Then there ex- 
ists a constant C depending only on Xo,n,r,e,h and L such that 





(4.1) 



{exp(XA t — $ r (\B t )),te T} 

is a super martingale with mean < 1 for < A < Aq 



(4.2) 
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Proof. It suffices to prove (4.2) with sup t>0 replaced by sup s>f>0 for 
every s > 0. Given any s > 0, there exists a sequence of nonnegative random 
times r n < s (in general, not stopping times) such that 

A + 

lim — 

^ n^co (B Tn V ri){\ V log+ L(B Tn V i])}^ 1 )^ 



At 



oS< s (B t V 7?){1 V log+ I(fl t V 77)} (T " -1)/r ' 

since ^4o = 0- As in the proof of Theorem 3.6, we shall assume without loss 
of generality that Bt > rj. Take any q < 1 such that gAo?] > e. 

It follows from Lemma 1 of Shao (2000) and Fatou's lemma that for any 
nonnegative supermartingale {Yt,t G T} (with right-continuous Yt in the 
case T= [0,oo)), E(sup teT Y t ) q < (1 - g) _1 (-E>o) 9 - Applying this result to 
(4.1) and noting that Aq = 0, we obtain that for < A < Ao, 

(1 - q)" 1 > £(supexp{AA t - $ T (\Bt)}Y 

\t£T / 

(44) >Eexp{q[\Ar n -* r (\B Tn )]} 

= Eexp{q\A Tn - V r (q\B Tn )}, 

where ^ r (x) = q 1 ~ r x r jr. 

Let f r (w) = exp{g(l — r~ 1 )w r ^ r ~ 1 ^} for w > 0. Note that in the notation 
of Theorem 3.6, g^ r (w) = y~ 1 f r (w) with y w = qw 1 ^ 1 " -1 ^. Letting A = A Tn 
and B = B Tn , it follows from (4.4) and (3.3) that 

which in turn yields the following analogues of (3.19) and (3.21), with r\ > rj: 
(4.5) (l- g )-i>___Iog(5U 



SAi^Vrj) Vr// I(S VI)' 

(4.6) 



[ q) ~ 1 xL(x/(qB) V (qB)/x) 



> c(X ,q,v,r)E - J^l^- - 1 (Y-V^ ^ < A £ 

with gAo?? > e, Ai = Ao V Aq' 1 and the constant c(Ao, q, rj, r) depending only on 
Xo,q,rj and r. For (4.6), recall that y w = qw 1 /^ -1 ' and g^ r (w) = y~ 1 f r (w). 

Take any 5 < 1 such that r(l — S)/(r — 1) > 1. Since qXofj > e, there exists 
xq > Aq" 1 V 1 such that 

(4.7) h(x) < e qX ^ x /L{x) < f^~ 5 (x)/x 1 ^ r - 1 ^ for all x > x , 
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noting that L(x) < 3xL(l) by (3.1). Let 
F = {f*(A/B)<L{B)Ve} 

= {A+/B < [(1 V logL(B))/(Sq(l - r- 1 ))]^" 1 ^}. 
Let k be the smallest integer such that 2 k (r — 1) > 1. On {A/B > xq V 

(Ao^r 1 }, 

L(A/B) > L(x V (Ao^)^ 1 ) > |(A A l) r - 1 L(l V B^ 1 ) 
>3~ (fc+1) (A Al) r " 1 L(l V5), 

where the last two inequalities follow from (3.1) and (3.2), respectively. 
From (4.7), it then follows that 

( £ ) 

\B{1 Vlog+L(B)}(r-V/rJ 

< h(x ) + h(l/[5q(l - r-^-^PiF) 

+ £i(Vn{^>x 

(4.8) t MovA/B //yn 1 /^-!) 

-1(1-1 >X B 



Eh 



3-( fc+1 )(A Al^BVl) KKB, 

( h{x)x l '^ \ f r (A/B) 
{!>l fy\x) J(A/By/^i) L{B) 



-A\ V(»-i) 
xlll-l <A B 



noting that f*{A/B) > L{B) and, therefore, ft-* (A/B) < f r (A/B)/L(B) 
on F c . The desired conclusion follows from (4.5), (4.6) and (4.8). □ 

Consider the case of continuous local martingales At- We can apply Theo- 
rem 4.1 with r = 2 and Bt = \J {A)t, in view of Lemma 1.2. Putting h(x) = x p 
in (4.2), with L(x) given by (3.4) in this case, yields the following exten- 
sion of (1.3) to the case q = p: There exists for every p > an absolute 
constant C p such that 

(4 - 9) ^m^kv^w)'- c '- 

Since (4.1) holds for all Ao > by Lemma 1.2, we can, in fact, set Ao = oo 
in (4.8) with r = 2 to replace it by 

4 + 



B{\ Vlog+L( J B)} 1 /2 
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<h(x ) + h([2/5q] 1 / 2 )P(F) 



x >^ exp{(( Z /2)(l-5)x 2 } V IS" U Jy (A/B)L(B) 

so we only require < exp(ex 2 ) for some e < \ and all large x in this 
case. Putting h{x) = exp(ax 2 ), with < a < ^, in the preceding argument 
then yields an absolute constant C(a) such that 



(4.10) E 



sup cxp 



aA 



i>£ "V(A) 4 loglog((^)iV e 2; 



<C(a), 



which can be regarded as an extension to p = of the following result of 
Kikuchi (1991): For every p > and < a < h, there exists an absolute 



constant C QjP such that 



E[^exp(a^/(A) 00 )] < C a , p E{A^ 



00/ ' 



where ^ = sup f > |^ t |. 

By Lemma 1.5 or 1.6, (4.9) (with t > replaced by n > 1) also holds for 
discrete-time supermartingales or martingales A n whose difference sequences 
satisfy the assumptions in these lemmas. Similarly, for conditionally sym- 
metric random variables dj, it follows from Lemma 1.4 and Theorem 4.1 
that for every p > 0, there exists an absolute constant C p such that 

(4 ' 11) Kg «E^)Jfcgl*v^}*0 ^, 

In view of Lemma 3.9(iii), Theorem 4.1 can be applied also when {d n ,J- n , n > 
1} is a super martingale difference sequence such that d n > — M a.s. for all n 
and some nonrandom M > 0. In this case, we have more generally that for 
p > and 1 < r < 2, there exists C PjT . such that 

(412) HSS {(E?=i Mil- V l)[loglog(ELi W V e»)]-i}i/r J 

The remainder of this section considers maximal inequalities for self- 
normalized processes under condition (4.1) by using an extension of the 
method of mixtures introduced by Robbins and Siegmund (1970) for Brow- 
nian motion. Let F be any finite measure on (0, Ao) with F(0, Ao) > and 
define the function 

(4.13) 4>(u, v) = [ X ° exp{Au - X r v/r} dF(X). 



Given any c > and v > 0, the equation ip(u,v) = c has a unique solution 
u = 0f{v, c). For the case r = 2, the function v — ► 0f(v, c) is called a Robbins- 
Siegmund boundary in Lai (1976), in which such boundaries are shown to 
have the following properties: 
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(a) Pf{v,c) is a concave function of v. 

(b) lim^oo Pf(v,c)/v = 6f/2, where 6p = sup{y > : F(0, b) = 0}(sup = 

0). 

(c) If dF(\) = f(X) dX for < A < A and inf 0<A<Ao /(A) > while sup 0<A<Ao /(A) < 
oo, then f3p(v,c) ~ (vlogv) 1 ^ 2 as u — > oo. 

(d) If dF(X) = f(X) dX for < A < e~ 2 , and = elsewhere, where 

(4.14) /(A) = lAAOogA^XloglogA- 1 ) 1 ^}, 

for some 5 > 0, then as i> — > oo, 



1/2 

(4.15) p F {v,c) = \2v 



log 2 u + Q + log 3 v + log (^7=j + 

As in Robbins and Siegmund (1970), we write log fc v = log(log fc _ 1 v) for k > 
2,log 1 f = logv. For general 1 < r < 2, (a) still holds, (b) holds with &f/2 
replaced by b r f l jr and (c) can be generalized to (3f(v,c) ~ i> 1//r {(logi>)/(r — 
l)}( r-1 )/ r as v — > 00. Moreover, if / is given by (4.14) as in (d), then 

(4.15) /?H^c)~t> 1/r {r(loglogt>)/(r-l)} (r - 1)/r as v -> 00, 

as can be shown by a modification of the arguments in Section 5 of Robbins 
and Siegmund (1970) for the case r = 2. 

It follows from (4.1) that {ip(At, B%),t > 0} is a nonnegative supermartin- 
gale with mean < F(0, Xq) and, therefore, 

P{A t > Pf(BI, c) for some t > 0} 

(4.16) 

= P{ip(A t ,Bl) > c for some t > 0} < F(0, A )/c, 

for every c > 0. In particular, by choosing c in (4.16) arbitrarily large, we 
obtain from (4.15) and (4.16) the following: 

Corollary 4.2. Let 1 < r < 2, $ r (x) = x r jr for x > and suppose that 
(4.1) holds for the process (A t ,B t ), t&T. Then 

^ At < f r \ ( r " 1 )/ r 

™ « P B t (log log B t ) ~ U^l) 

(4.17) 

a.s. on < lim Bt = 00 

\t— >oo 

Note that Theorem 4.1 already implies the a.s. finiteness of the above 
lim sup on {limi?^ = 00}, but (4.17) gives a sharp nonrandom upper bound 
that reduces to the familiar \[2 when r = 2. In view of Lemma 1.4, Corol- 
lary 4.2 with r = 2 is applicable to conditionally symmetric random vari- 
ables di, yielding (4.17) with A t = £*=i di and B t = (£* =1 d i ) 1/2 - The spe- 
cial case of this result for independent symmetric di has been derived via an 
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independent Rademacher sequence {ei} by Griffin and Kuelbs (1991) and 
also by Gine and Mason (1998), who show that loglogl^ in (4.17) (with 
r = 2) can be replaced by log log t when the d{ are i.i.d. symmetric. 

We next extend the preceding method of mixtures to derive maximal in- 
equalities for conditionally symmetric mxl vectors. An adapted sequence of 
random vectors {di} is called conditionally symmetric if {X'd{\ is an adapted 
sequence of conditionally symmetric random variables for every A E R m . By 
Lemma 1.4, if {d{\ is a sequence of conditionally symmetric random vectors, 
then for any probability distribution F on R' m , the sequence 



(4.18) 



,. ( n n \ 

/ expi A'^^-±A'^(f^AUF(A), n > 1 
■^ Rm I i=i i=i J 



forms a nonnegative supermartingale with mean < 1, noting that (X'di) 2 = 
X'didjX. In particular, if we choose F to be the multivariate normal distri- 
bution with mean and covariance matrix V , then (4.18) reduces to 



\V\ 1/2 



v+j2did[ 



i=l 



expj(X>^ ^V+J^m) (l> 



(4.19) 

where | • | denotes the determinant of a square matrix. Hence, for any c > 
and any positive definite m x m matrix V, 

p { (S"=i d'j){V + E"=i (kd'j)- 1 (YA=idi) > 1 for gome n>1 U e - lm 
\ \og\V + YA =1 di%\ + 2\og(c/y/\7\) - - J- 

(4.20) 

As another application of the method of mixtures, we derive a simple for- 
mula for certain boundary crossing probabilities of multivariate continuous 
local martingales. Let A m i n (-) denote the minimum eigenvalue of a nonneg- 
ative definite matrix. 



Corollary 4.3. Let Mt be a continuous local martingale taking val- 
ues in R m such that Mq = 0, lim^oo A m i n ((M) t ) = oo a.s., and such that 
E exp(X' (M)tX) < oo for all X £ R m and t>0. Then for any c > 1 and any 
positive definite m x m matrix V , 

(4.21) P< ■ ■ — — — - n — . > 1 tor some t> > = c , 

v ; \log|V+<M) t |+21og(c/v/IVT) - J 

Proof. First note that an expression similar to (4.19) is equal to the 
integral 

(4.22) J exp(x'M t - ^X'{M) t x} dF(X), 
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where F is the m-variate normal distribution with mean and covari- 
ance matrix V -1 . Given any A G R m with A 7^ 0, X'Mt is a univariate lo- 
cal martingale and (X'M) t = X'(M) t X — ► 00 a.s. since X m i n ((M) t ) — > 00 a.s. 
Hence, by the martingale strong law, X'Mt/X' (M)tX — > a.s. and, therefore, 
exp{A'M t - (X'M) t /2} -> a.s. as t -> 00, for every A / 0. 

Since E exp({X' M)t/2) < 00, it follows from Novikov's criterion [cf. Revuz 
and Yor (1999), page 332] that {exp(A'M 4 - (X'M) t /2, t > 0} is a martin- 
gale. Therefore, / exp{A'M 4 — (X'M)t/2}dF(X) is a nonnegative continuous 
martingale, and by Doob's inequality, the probability in (4.21) is < c _1 , sim- 
ilar to (4.20). Equality actually holds in (4.21), by Lemma 1 of Robbins and 
Siegmund (1970), if it can be shown that (4.22) converges to a.s. as t — > 00. 
Since exp{A'Mf — (X'M) t } — > a.s. for every A 7^ 0, we need only apply the 
dominated convergence theorem and note that by Doob's inequality, 

P j J exp(A'M t - (X'M) t /2) dF(X) > c for some t > J 
<c^/ dF(X). 

J\\X\\>a U 

5. A universal upper LIL. To derive (1.7) for any adapted sequence 
{Xi}, one basic technique pertains to upper-bounding the probability of 
an event of the form = {tfc-i < T k < tk} m which tj and Tj are stop- 
ping times defined in (5.3). Sandwiching between t^-i and t/. enables us 
to replace both the random exceedance and truncation levels in (5.3) by 
constants. Then the event E^ can be re-expressed in terms of two simulta- 
neous inequalities, one involving centered sums and the other involving a 
sum of squares. Using these inequalities, we derive a supermartingale that is 
then used to bound P(Ek). Apart from finite mean constraints, Lemma 5.1 
gives the basic idea underlying the construction of this supermartingale. It 
will be refined in Corollary 5.3 to enable us to remove the assumptions in 
Lemma 5.1 concerning both the integrability of the Y^s and the restrictions 
on the negative part of their support. 

Lemma 5.1. Let < 7 < 1 and define 

00 

(5.1) C 7 = -{7 + log(l - 7 )}/7 2 = E 7 j ~ 2 /i- 

Then C 7 = = c 7i 2, where c 7>r and cp^ are the same as in Lemma 3.9. 
Moreover, if Y is a random variable such that Y > —7 and E\Y\ < 00, then 
Eexp{Y -EY - C^Y 2 } < 1. 

Proof. As shown in Lemma 3.9(i), exp(y — C^y 2 ) < 1 + y for all y > —7. 
Hence, Eexp{Y - C^Y 2 } < 1 + EY < exp(EY). □ 
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Corollary 5.2. Fix any < 7 < 1. Let {J- n } be an increasing sequence 
of a -fields. Suppose Y n is T n -measurable, E\y n \ < 00 and Y n > —7 a.s. Let 
fi n = E(Y n \!F n -i) . Then exp{^™ = i(^ — fJ>i — C^Y 2 )} is a supermartingale 
whose expectation is < 1. 

Corollary 5.3. Let {.F n } be an increasing sequence of a-fields and 
Y n be J- n -measurable random variables. Let < j n < 1 and < A n < 1/C 7n 
be T n ^\-measurable random variables, where C 7 is defined in (5.1). Let 
H n = E{Y n t{- ln < Y n < A n )|^_!}. Then exp{E?=i(^ - IH~ K% 2 )} is 
a supermartingale whose expectation is < 1. 

Proof. Observe that exp{y — y 2 /Xi} < 1 if y > \ or if y < — 7^. Let 
Xi = Fil(-7i <Yi< Xi). Then " 

^{exp^-^-Ar 1 ^ 2 )!^!} 

< ^{exp^ -fa- X' 1 Xf)\Ti-i} 

< E{(1 + X^e-^-!} = (1 + Mi)e -W ! 

see the proof of Lemma 5.1 for the last inequality, recalling that //, = 
E(Xi\J r i-i). Since (1 + x)e~ x < 1 for all x, the desired conclusion follows. 

□ 

The centering constants in (1.7) involve sums of expectations conditioned 
on the past which are computed as functions of the endpoints of the interval 
on which the associated random variable is truncated. The actual endpoints 
used, however, are neither knowable nor determined until the future. Thus 
the centered sums that result are not a martingale. Nevertheless, by using 
certain stopping times, the random truncation levels can be replaced by 
non-random ones, thereby yielding a supermartingale structure for which 
Corollary 5.5 applies, enabling us to establish the following result. 

Theorem 5.4. Let X n be measurable with respect to J- n , an increasing 
sequence of a-fields. Let A > and h(X) be the positive solution of 

(5.2) h-\og(l + h) = X 2 . 

Let b\ = h(X)/X, 7 = h(X)/{l + h(X)} and a\ = X/ (7C 7 ), where C 7 is defined 
by (5.1). Then (1.7) holds on {linin^oo V n = 00} and lim^o^A = V%- 

PROOF. Recall that V 2 = X 2 -\ \-X 2 and v n = y n (loglogK)~ 1/2 - Let 

efc = exp(£;/log &). Define 

tj = inf{n : V n > ej}, 

(5.3) r 1 

Tj =Ml n> tj:S n - 22 fii(-Xv n ,axv n ) > (1 + 3e)6 A V r „(loglogK) 1/2 |, 
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letting inf = oo. To prove (1.7), it suffices to show that for all sufficiently 
small e > 0, 



(5.4) 



lim P{T k <t k+1 } = 0. 



K^oo 



k=K 



Note that r k > t k and that t k may equal t k+ i, in which case {r k < t k+ i} 
becomes the empty set. Moreover, on {lmin^oo V n = oo}, tj < oo for every 
j and limj^ootj = oo. Since ?/(loglog?/) _1 / 2 is increasing in y > e%, we have 
the following inequalities on {t k < r k < ifc+i} with k > 3: 

fr k \ 1/2 

(5.5) e k < (J2 X ?) < e k+i, 

(5.6) d k := e fc (logloge fe )~ 1/2 < v tk < v Tk < d k+1 , 

(5.7) (i i (-\v Tk ,axv Tk ) > m(-Xdk+i,axdk) forl<i<r fc . 

Let fj,i ; k = fJ-i(—Xd k+ i,axd k ). We shall replace Xi (for 1 < i < r k ) by Yi k '■= 
(AcZ fe+ i) _1 7Xj and fj, ik by fa k ■= (Ac4+i) _1 7Mi(-Mfc+i) axd k ). Since A _1 7«A 

(5.8) fk, k = E{Y i)k l(--y < Y i>k < C-^fc/dfc+OI^-!}. 

Since e k /d k = (log log e^) 1 / 2 and d k /d k+ i — ► 1 as k —> oo, it follows from 

(5.5)-(5.7) that for all sufficiently large fe, the event < r k < t k+ i} 
is a subset of 

|^(A4+i) -1 (Xj - /Xj )fc ) > (1 + 2e)A~ 1 fe A logloge fc , r fc < ool 



.i=l 



c \ ^[(A4+i) -1 7(^ - fH,h) ~ C 7 (4+i/4)(A4+i)~ 2 7 2 X 2 ] 
u=i 

> (1 + 2e)7A" 1 6 A logloge fc - C 1 (d k+1 /d k )('y/X) 2 logloge fc+1 , r fe < oo 



C < supexp 

n>l 



y^(^i,fc — fii,k — C 1 d k d k+ iY 2 k 



i=l 



> exp[(l + e)( 7 A- 1 6 A - C 77 2 A- 2 )(logfc)]|. 

In view of (5.8), we can apply Corollary 5.3 to conclude that the last event 
above involves the supremum of a nonnegative supermartingale with mean 
< 1. Therefore, application of Doob's inequality to this event yields 

P{r k < t k+l } < exp{-(l + e)( 1 X- 1 b x ~ C^ 2 X~ 2 )(logk)}, 
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which implies (5.4) since 

(5.9) tA- 1 ^ - A- VC 7 = A^ 2 { 7 /!(A) + 7 + log(l - 7)} = 1- 

The first equality in (5.9) follows from (5.1) and 6a — h(X) /A, and the second 
equality from 7 = h(A)/(l + h(X)) and (5.2). Moreover, (5.2) implies that 
h 2 {\) ~ 2A 2 and, therefore, 6 A -> \/2 as A -> 0. □ 

Remark 5.5. The choice of 7 in Theorem 5.4 actually comes from min- 
imizing 7A~ 1 &a — A~ 2 7 2 C 7 over < 7 < 1, whereas 6a is employed to make 
this minimizing value equal to 1, leading to the equation (5.2) defining h(X). 

As pointed out in Section 1, an immediate consequence of Theorem 5.4 
is the upper half (1.7) of the LIL for any supermartingale whose difference 
sequence X n is bounded below by — Xv n . The following example shows that 
we cannot dispense with this boundedness assumption. 

Example 5.6. Let X± = X2 = 0, X%, X4, ... be independent random vari- 
ables such that 

P{X n = -n~ 1/2 } = 1/2 - n~ 1/2 (logn) 1/2 - n-^logn)" 2 , 

P{X n = -m n } = n-^logn)" 2 , P{X n = n" 1 ' 2 } = 1/2 + n~ l l 2 {\ogn)^ 2 

for n > 3, where m n ~ 2(logn) 5//2 is chosen so that EX n = 0. Then P{X n = 
—m n i.o.} = 0. Hence, with probability 1, V 2 = X}£=i^ _1 + 0(1) = logn + 
O(l). Since Xi := Xi±(\Xi\ < 1) - EXit(\Xi\ < 1) are independent bounded 
random variables with zero means and Var(Xj) Kolmogorov's LIL 

yields 

(5.10) limsup(y"Xi J /{2(log?i)(logloglog?i)} 1/2 = 1 a.s. 

\i=i )' 

Since Y J r }=xEX i t{\X i \ < 1) ~ 2 Yd=i i _1 (logi) 1/2 ~ f (logra) 3 / 2 , this implies 
that with probability 1, 

T,i=iXi s"=i^a(l^l<i) 

7 — rNw ' T~ 

V n (log log K) 1/2 { (log n) (log log log n) j 1 ' 2 

4 (logn) 3 / 2 



00. 



3{(log n) (log log log n)} 1 / 2 

Note that ?n n (loglogHi) 1//2 /^ — > 00. This shows that without the bound- 
edness condition X n > — AV n (loglog V„) _1//2 , the upper LIL need not hold 
for martingales self- normalized by V n . It also shows the importance of the 
centering in Theorem 5.4 because subtracting < 1) from Xi gives 

the LIL in view of (5.10). 
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Note that Corollary 5.3, which leads to Theorem 5.4, only uses the spe- 
cial case r = 2 of Lemma 3.9(i). More generally, for 1 < r < 2, we can use 
Lemma 3.9(i) and the same arguments as those in Lemma 5.1 and Corol- 
lary 5.3 to show that 

exp {£(Y, - Eim-H <Yi< Aj /(r_1) )|.Fv-i] - Ar 1 ^)}, 
(5.11) U=i J 

n > 1, is a supermartingale, 

for any .T^i-measurable random variables < 7$ < 1 and < Aj < l/cj r , 
where c 7jr is defined in Lemma 3.9. Therefore, Theorem 5.4 can be extended 
to the following: 

Theorem 5.7. Let X n be measurable with respect to J- n , an increas- 
ing sequence of a-fields. For 1 < r < 2, let V n>r = (Ya=i |- 5 Q| r ) 1//r , v n>r = 
V^ ir {loglog(F njr V e 2 )} _1//r . Then for any < 7 < 1, there exists a positive 
constant 6 7>r such that 

limsup|s n - f^Mi(-7^r,C 7 - 1 / (r - 1 ^ n , r ) 1 /{K ! r(l0gl0gK,r) (r-1)/r } 
n^oo [ . =1 J 

< 6 7jT . a.s. 

on {limn^ooVn^ = 00} , where c 7ir is gi^en in Lemma 3.9. 

6. Compact LIL for self-normalized martingales and applications to sums 
of independent random variables. Although Theorem 5.4 gives an upper 
LIL for any adapted sequence {X{\, the upper bound in (1.7) may not be at- 
tained. A simple example is S n = J27=l w i^i-> where Wi = i! and Y\, Y2, ■ ■ ■ are 
i.i.d. with P{Yi = 1} = \ = P{Yi = -1}. Here V n = (£?=i v$Y?)V 2 ~ n\, 
S n /V n = sgn(Y n ) + o(l) and £™ =1 /^(-Ai; n , a A v n ) = o(V n ) a.s. Thus, the 
norming term ^(loglog V^) 1 / 2 is too large in this case. In this section we 
consider the case of martingales {S n , !F n , n > 1} self-normalized by V n and 
prove the lower half counterpart of (1.8) when the increments of S n do not 
grow too fast, thereby establishing (1.9). This is the content of Theorem 6.1, 
which is further strengthened into a compact LIL in Corollary 6.2. We end 
this section with an application to weighted sums (with random weights) of 
i.i.d. random variables, a remark on Theorem 5.4 and an example highlight- 
ing the difference between this LIL and an analogous LIL of Stout (1970) in 
which is replaced by s 2 = Y2=i E ( X i\fi-i)- 

Theorem 6.1. Let {X n } be a martingale difference sequence with re- 
spect to an increasing sequence of a-fields T n such that \X n \ < m n a.s. 
for some T n -\-measurable random variable m n , with V n — ► 00 and m n / 
{K(loglogK)~ 1/2 } -+0 a.s. Then (1.9) holds. 
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Proof. Take <b < <0 <y/2. Since 1 - = exp{-(± + o(l))x 2 } 
as x — ► oo, we can choose A sufficiently large such that 

(6.1) {l-d>(/VA)} 1/A >exp(-/3 2 /2), 

where <3? is the standard normal distribution function. Take a > 1 and define 
for j > 2 and A; = 0, 1, ... , [A -1 logj], 

a jife = a' + k(aJ +1 - a^/IA" 1 logj], ^-(fc) = inf{n : V* > a jtk }. 

Let tj = inf{n: F n 2 > a J '}, so tj(0) = tj, ^([A" 1 log j]) = t j+1 . Since X 2 = 
°(^n OoglogK)" 1 ) a.s. and a jtk < V^ {k) < a jtk + X^ k y 

(6.2) V t ) {k) = a hk {l + ((log j)- 1 )} a.s. 
It will be shown that 

E x ll E ^Mjvo-i 

g-j t,(fc)<n<t,(fc+l) tj(k)<n<tj(k+l) 

in probability under -P(-|^(fc)) 

as j — > oo, uniformly in < k < [A" 1 logj]. 

Let S m>n = Em<i<n X i, V m,n = Em<i<n X i ■ In view of ( 6 - 2 )> 

(M V^ {k)Mk+1) ^a\a-l)/[\- 1 \ogj], V* t . +1 ~a>'(a- 1) a.s. 

Since X 2 is bounded by the ^" n -i -measurable random variable m 2 , which 
is ©(V^loglog V^)" 1 ) a.s., the conditional Lindeberg condition holds and, 
in view of (6.3) and (6.4), the martingale central limit theorem [cf. Durrett 
(1996), page 414] can be applied to yield 

(6.5) P{S tj{k)Mk+l) > Py/\V tj{k)Mk+1) \F t . {k) } - 1 - *(pV\) a.s. 
as j — > oo, uniformly in < k < [A -1 logj]. Since 

S tj,tj+i = E S t : 



0<fe<[A- 1 logj] 



and 



^ |tj+1 (logj) 1/a = (VA + (l)) E V t 

0<fc<[A" 1 logi] 
by (6.4), it follows from (6.5) that as j — ► oo, 
P{S tj>tj+1 >bV t . ;tj+1 (logj) 1 / 2 \F tj } 

> P{S tj[k)Mk+1) > 0V\V t . ik)Mk+1) for all < k < [A" 1 log j\\F t .} 

= (l-$(/VA)+o(l)) [A ~ llogj] 
>exp{-(/3 2 /2 + o(l))logj} a.s., 
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in view of (6.1). Since /3 2 /2 < 1, the conditional Borel-Cantelli lemma then 
yields 

(6.6) limsup5 t ,,i, +1 /{^,t J+1 (logi) 1/2 }>& a.s. 

j->oo 

Recalling that V n — > oo and m n = o(V^(loglog V n )~ 1 / 2 ) a.s., we obtain 
from (1.8) that 

(6.7) limsupS n /{y n (loglogy n ) 1/2 }<V2 a.s., 

n—*oo ' 

and the same conclusion still holds with S n replaced by — S n (which is a 
martingale). Combining this with (6.4) and (6.6) yields 

lim sup S tj+1 /{V tj+1 (log log V tj+1 ) 1/2 } 

j— >oo ' 

(6.8) 

>ba- l l\a- if' 2 -V2a- 1 ' 2 a.s. 
Since a can be chosen arbitrarily large and b arbitrarily close to \/2 in (6.8), 

hmsup^ j+1 /{^ +1 (loglog^ J+1 ) 1/2 } > V2 a.s. 

Combining this with the upper half result (6.7) yields (1.9). 

It remains to prove (6.3). Let ctj = a J (a — 1)/[A _1 logj']. In view of (6.4), 
we need to show that given any < p < ^ and 5 > 0, 



lim sup 
(6.9) 



Pi ]T ^(x^O^i + pK- 

ltj(k)<n<tj(k+l) 



+ P< 



ft 



ft 



<5 



a.s. 



U.,(fc)<n<t,'(fc+1) 

Choose e > such that 2{max[(l + p)e~^, (1 - p)e p ]} 1/e < 5. Let X n = X n t{m 2 n < 
eotj) and note that since m n is JF n „ 1 -measurable and X 2 < m 2 , 

< E{X 2 n \T n ^) - E{X 2 n \T n -x) < m 2 n l(m 2 n > eaf). 
Moreover, P{m 2 l < eotj for all tj(k) < n < tj(k + l)|^(jfe)} — ► 1 a.s. Hence, 
it suffices to consider E(X 2 \F n -i) instead of E(X 2 \J r n ^i) in (6.9). Since 
X 2 < ectj, we can apply Corollary 15 of Freedman (1973) to conclude that 

P\ E ^|^n-l)>(l+pH|^( fc )l 

ltj(k)<n<tj(k+l) ) 



+ Pl E E(X 2 \T n ^)<(l-p) aj 

ltj(k)<n<tj(k+l) 

< (1 + p)e- p l £ + (1 - p)e p / £ + o(l) < 8, 



ftt^k) 
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completing the proof. □ 

Corollary 6.2. With the same notation and assumptions as in Theo- 
rem 6.1, the cluster set of the sequence {S n /[V n (loglog(V n V e 2 )) 1 / 2 ]} is the 
interval [— y/2, \/2]. 

Proof. Replacing X n by — X n in Theorem 6.1 yields liminfn^oo 

S n /{K(loglogK) 1/2 } = 
— \pl a.s. The desired conclusion then follows from Proposition 2.1 of Griffin 
and Kuelbs (1989). □ 

Example 6.3. Let Y\_,Y2, ... be i.i.d. random variables with a common 
distribution function F having mean 0. Let T n be the a-field generated 
by Yx,...,Y n . Let w n be T n -\ -measurable and let S n = Y^=i w iYi, V% = 
Y^i=\ w V^i ■ Suppose V n — > oo a.s. and there exists T n -\ -measurable m n 
such that with probability 1, 

(K(io g iogv;r 1/2 ), 
oCKaogiogv;) 1 / 2 ), 

o(K 2 ), 

(6.12) \ F{m n /\w n \) + F{-m n /\w n \) \ < oo, 

n=l ^ > 

where F(x) = P(Yi >x) = l- F(x-). Let X n = w n Y n t(\w n Y n \ < m n ). Then 
E(X n \F n -i) = — w n J\ WnX \> mn x dF(x). Moreover, by (6.12) and the condi- 
tional Borel-Cantelli lemma, with probability 1, 

n 

(6.13) w n Y n = X n for all large n and therefore V% = Xf + O(l). 

i=i 

Applying Corollary 6.2 to Yd=i{ x i - E(X i \F i ^ 1 )} (with < m;) and 
combining the result with (6.11) and (6.13), we obtain [— y2, \/2] as the 
a.s. cluster set of the sequence {S n /[V n (loglog(V n V e 2 )) 1 ' 2 ]}. Note in this 
connection that 

n 

Y^iXi-EiX^-!)} 2 

i=l 

n n n 

= J2 x f- 2 Y,i x i - £pQi^-i)}£pQi^-i) - xXPTii-^-i) 

i=l i=l i=l 



(6.10) 
(6.11) 



< m n 
^2 w i xdF(x) 

• -i J\wix\>rn,i 



E 

i=i 



8=1 



if. 



Wix\>rrii 



xdF(x) 



30 



V. H. DE LA PENA, M. J. KLASS AND T. L. LAI 



i=l i=l 

(/ n \ 1 / 2 / n \ V 2 \ 

( -^i-^-i)} 2 J f E^(^i^-i) J )• 

Note that Theorems 6.1 and 6.2 pertain to martingale difference se- 
quences X n . This means that given an integrable sequence {X n }, one should 
first consider centering X n at its conditional expectation given J- n -\ before 
applying the theorems to X n = X n - E{X n \J : n - 1 ) and V n = (Th=i X i) 1/2 - 
Although Theorem 6.1 requires X n to be bounded by .F^-i-measurable 
m n = o(y n (loglog V^)" 1 / 2 ), we can often dispense with such boundedness 
assumption via a truncation argument, as shown in Example 6.3. In the 
more general context of Theorem 5.4, the X n may not be even integrable, 
so Theorem 5.4 centers the X n at certain truncated conditional expecta- 
tions. Using (J2i=i Xf) 1 / 2 for the norming factor, however, may be too 
large since it involves uncentered Xi's. To alleviate this problem, we can 
first center X n at its conditional median before applying Theorem 5.4 to 
X n = X n — med(X n |^ r n _i), as illustrated in the following: 

Example 6.4. Let < a < 1, d x > 0,d 2 > with d\ + d 2 > 0. Let Y,Yi,Y 2 , , 
be i.i.d. random variables such that 

P{Y>y} = (d 1 + o(l))y- a , 

(6.14) 

P{Y <-y} = {d 2 + o{l))y- a asy^oo. 

Let S n = E?=iYi, V 2 = E? =1 3f, v n = K(loglog Vn)- 1 ' 2 . Then by Theo- 
rem 5.1 of Shao (1997), 

(6.15) \imsu V S n /{V n {\og\ogn) l ^} = {P{a,d 1 ,d 2 )}- 1 ' 2 a.s. 

for some positive constant (3 (a, di, d 2 ) which is given explicitly in his Theo- 
rem 3.2. Moreover, E{Yl(-\y < Y < a x y)} = (dia A -d 2 A + o(l))ay 1 - a /(l- 
a) as y — > oo and 

^"/{^.(loglogK) 1 / 2 } = n/{C(loglogV;)( 2 ^)/ 2 } = 0(1) a.s. 

(6.16) 

since log log V n ~ log log n and 

liminf (^Y^j / {n 1/& {\og\ogn)- {l - &)l& } > a.s. with a = a/2, 

by the so-called delicate LIL [cf. Breiman (1968)]. 

Now let X n = rf + Y n with r > 1/a and let S n = Ya=i X h = E?=i x h 
Since Y n = o(n s ) a.s. for any s > 1/a, it follows that S n ~ V n ~ n r+l /(r + 
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1) and Hi(—Xv n ,a\v n ) = V + o(n( r+1 ^ 1 ~ a ^) = i r + o(n r ) a.s., recalling that 
ra > 1. Therefore, although (1.7) still holds in this case, it is too crude 
as the nonrandom location shift n r is the dominant term in X n causing 
V n to swamp the centered S n . Centering the X n first at its median will 
remove this problem. Specifically, if we apply (1.7) to X n = X n — med(X n ) 
and V 2 = YA=iXh then X n = Y n - med(Y) and (6.15) still holds with S n 
replaced by S n . 

The following example shows that one cannot dispense with the assump- 
tions of Theorem 6.1 and highlights the difference between our result and the 
LIL of Stout (1970), where the martingale S n is normalized by the square 
root of the conditional variance Ya=i E(Xf\Fi-i). 



Example 6.5. Taking X\ =0,X2,X3,... and m n as in Example 5.5, 
let Y n = X n t(\X n \ < 1). Then P{Y n / X n i.o.} = P{Y n = -m n i.o.} = 0. As 
shown in Example 5.5, with probability 1, V 2 = Y^i=i + 0(1) = logn + 
0(1) and 

(61?) S? = i*t 4(logn) 3 / 2 

V n (log log Vn) 1 / 2 3{ (log n) (log log log n)} 1 / 2 

Note that m n (log log Vn) 1 / 2 /V n — > oo. This shows that without the condi- 
tion m n /{T4(loglogy n ) -1 / 2 } — > 0, the LIL need not hold for martingales 
self- normalized by V n . On the other hand, X n is clearly bounded above 
and, therefore, satisfies the boundedness condition of Stout (1970). Note 
that Var(Xi) ~ 4(logi) 3 /« and, therefore, s 2 n := ELi ^M^i-i) ~ (logn) 4 , 
yielding 

(618) EU*i 4(logn) 3 / 2 Q 

s n (log log Sn) 1 / 2 3(log n) 2 (log log log n) 1 / 2 

which is consistent with Stout's (1970) upper LIL. Contrasting (6.18) with 
(6.17) shows the difference between Stout's result and ours. Notice that what 
is being investigated in (6.17) is the maximal a.s. growth rate of S n . To assess 
it we employed a norming sequence based on the square root of its sum of 
squares. This technique works properly only when S n is adequately centered, 
as in (1.7). By contrast, in the approach of Stout, a norming sequence is 
generated from the square root of the sum of conditional expectations of 
these squares. However, in the absence of a suitable truncation of the random 
variables this quantity is also inappropriate for investigating almost sure 
behavior whenever expectations overinflate the impact of large values of the 
squares which occur too infrequently to be relevant with respect to almost 
sure behavior. 
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